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San Francisco's Bridge to Success (BtS) initiative brings together the City and County 
of San Francisco, the San Francisco Unified School District (SFUSD), the City College 
of San Francisco (CCSF), and key community organizations to promote postsecondary 
success for underrepresented students. Partners agree that the first step in achieving 
this goal is to ensure that more students are graduating from high school in a timely 
manner. One strategy that can be employed to accomplish this is the development of a 
set of indicators for an Early Warning System (EWS) that identifies students who are at 
risk of not graduating as early as eighth or ninth grade in order to provide them with 
extra supports to help move them back on track. 

Conversations between SFUSD staff and researchers from the John W. Gardner Center 
for Youth and Their Communities (JGC) at Stanford University have led to the 
proposed development of a three-part monitoring system that: 

• Identifies students at the end of the first semester of eighth grade who may benefit 
from participation in a summer program that helps them transition into high 
school; 

• Flelps high schools identify incoming ninth grade students who might benefit from 
additional supports; and 

• Monitors students during ninth grade to ensure that they stay on track for high 
school graduation. 

This research brief focuses on the development of indicators for the first two parts of 
the EWS. Analysis identified two eighth grade indicators that were most predictive of 
high school graduation - GPA and attendance rate - though the final cutoff points 
selected differed between the first semester of eighth grade and the end of the year. 
SFUSD and JGC have begun to co-develop a system of indicators for part three to 
ensure students stay on track during ninth grade. 

What is an Early Warning System? 

School districts around the country are developing Early Warning Systems (EWS), 
which primarily rely on academic indicators in eighth and ninth grade to identify 
students at risk of not graduating from high school. These data are generally used to 
identify students at one point in time (e.g. the first day of ninth grade) with the 
intention of providing targeted interventions early enough in students' academic 
trajectories to make a difference in their likelihood of graduating from high school. 
Additionally, EWS can be used to monitor a school's (or district's) performance over 
time in preparing students to enter high school on-track for graduation. 
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Several published reports have highlighted that academic characteristics, especially GPA and course 
failures, are the strongest predictors of high school completion, and adding demographic characteristics 
(gender, ethnicity, family background) generally offers little additional predictive power (Allensworth & 
Easton, 2005; Balfanz & Neild, 2006; National Research Council and National Academy of Education, 2011). 
Though an EWS can capture a large proportion of eventual dropouts, sometimes as many as 80% by the 
end of ninth grade, studies also recognize that many students, especially those who have reached tenth 
grade without exhibiting any conventional warning signs, will not be captured by these systems. One key 
reason is that some of the most important predictors of academic success (e.g. student motivation and 
tenacity, family involvement, instructional quality, school climate) are either inconsistently or not collected, 
or cannot be applied at the individual level to predict graduation rates. 


Exhibit 1. EWS Indicators Developed by a Sample of School Districts 


District 

Indicators Developed 

Chicago Public Schools 
(Allensworth & Easton, 2005) 

9 th grade "on-track" indicator: (1) Having one or no semester F grades in core 
courses; (2) Earning five or more credits freshman year (minimum needed to be 
promoted to 10 th grade). 

School District of Philadelphia 
(Balfanz & Neild, 2006) 

8 th grade indicators: (1) Attendance less than 80% and/or (2) Failed English or 
math. 

Portland Public Schools (Stid, 
O'Neill, & Colby, 2008) 

Focused on "Academic Priority Students" (D or F in a core course in 9 th grade or 
failed to meet proficiency in "two out of three standards") and "late entrants" 
(entered the district in 10 th grade or later). 

Baltimore City Schools 
(Baltimore Education Research 
Consortium, 2011) 

6 th grade characteristics: (1) Missing 20 or more days of school, (2) Failing English, 
or math, or both and/or a failing average for English, math, science, and social 
studies, (3) Being at least one year overage (suggesting an earlier retention), and 
(4) Being suspended for three or more days. 


Note: Some districts examined the same indicators across multiple grade levels, not just those specified here. This list is not 
intended to be exhaustive, for further information see (National Research Council and National Academy of Education, 2011). 


Helping High Schools Identify Students Requiring Extra Support 

Most students enter high school with a background that consists of both their schooling history, (e.g. GPA, 
standardized test scores, attendance, disciplinary behaviors, mobility) and other non-school based 
characteristics (e.g. socio-economic status, neighborhood or zip code, ethnicity, parent's education level). 
Developing an EWS requires identifying the age or grade at which the district is interested in intervening 
and then, using some set of the factors described above, identifying which are most predictive of high 
school graduation (or any other desired outcome) and what threshold or cutoff should be used for each of 
these variables (i.e. if previous GPA best predicts high school graduation, at what GPA should we begin to 
identify students?). This large set of information is then synthesized into a few simple rules that allow 
practitioners, including principals, teachers, counselors, and other staff, to identify students as needed and, 
ideally, provide them with an appropriate set of services or supports. 

This analysis followed two cohorts of students who were enrolled as first-time ninth graders during the fall 
semester of the 2005-06 and 2006-07 school years. All students were also enrolled as eighth graders both 
semesters during the previous school year. For the rest of this report we average data between these two 
cohorts, which included 3,295 and 3,469 students respectively, to give SFUSD a picture of how indicators 
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would impact a typical school year. This results in a cohort of an average of 3,382 students, who had a 
four-year high school graduation rate of 72. 9%. 1 

Regression analysis highlighted the importance of academic characteristics - including GPA, attendance 
rates, and California Standards Test (CST) scores - in predicting high school graduation rates. 2 Using a 
similar methodology to the School District of Philadelphia, we selected the final indicators only if they 
individually predicted a four-year high school graduation rate of approximately 25%, with the goal of 
creating a parsimonious system that utilizes only the most predictive variables (Balfanz & Neild, 2006). 
This resulted in the selection of two key eighth grade indicators: (1) GPA below 2.0 and (2) attendance rate 
below 87.5%. 3 Other predictive variables were eliminated as they did not meet the 25% threshold. 

A total of 749 students (22% of the incoming class of eighth grade SFUSD students) entered ninth grade 
with one or two risk factors. 4 Exhibit 2 illustrates that incoming ninth grade students who entered with 
exactly one risk factor were half as likely to graduate in four years as students who entered with zero risk 
factors (42.6% compared to 84.4%). The graduation rate of students with two risk factors dropped to 14.8%. 
Of the students with exactly one risk factor, 57.6% (n=274) had a low GPA and 42.4% (n=202) had a low 
attendance rate; students with low GPA were also slightly less like to graduate than students with low 
attendance (38.9% compared to 47.5%). 


Exhibit 2. Four-Year High School Graduation Rate, by Number of Year-End Eighth Grade Risk Factors 



1 This graduation rate is higher than the overall SFUSD graduation rate as it removes ninth graders who were not enrolled 
in the fall semester (entered late), entered SFUSD for the first time in ninth grade, and mobile students who were not 
enrolled both semesters of eighth grade. Students were defined as enrolled in the fall semester if they were enrolled on 
CBEDS date in mid-October and in the spring semester if they were enrolled on the last day of school. 

2 Other academic characteristics that were either inconsistently significant (significant in either 2006 or 2007 but not both 
years) or statistically significant with a small magnitude included entering ninth grade overage (i.e. over 14.75 years old) 
and being a long-term low-performing English Learner student (i.e. two to three consecutive years of California English 
Language Development Test proficiency at Beginner or Early Intermediate levels). 

3 Analysis used SFUSD' s instructional attendance rate, which is a measure of total missed classroom time. Low GPA and 
low attendance predicted high school graduation rates of 27% and 29% respectively, with only slight variation between 
the two years. We expect that year to year variation will exist and that the indicators presented in this analysis should be 
monitored over time to test their validity as new information becomes available. 

4 Approximately 85% of first-time ninth grade students were enrolled in SFUSD as eighth graders the previous year. Of 
those students who were not enrolled in SFUSD in eighth grade, almost half attended Lowell, Lincoln, or Newcomer. 
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Notes: The two risk factors were eighth-grade: (1) GPA below 2.0 and (2) instructional attendance rate below 87.5%. * Of the 
students with exactly one risk factor, 57.6% (n=274) had a low GPA and 42.4% (n=202) had a low attendance rate. Students with 
low GPA were slightly less likely to graduate than students with low attendance (38.9% compared to 47.5%). 
African-American, Latino, and male students were significantly more likely to enter ninth grade with at 
least one risk factor, but the prevalence of each risk factor varied significantly among groups (Exhibit 3). 
For example, males with one risk factor were much likely to have a low GPA and females were more likely 
to have low attendance. 


Exhibit 3. Percentage of Students With Risk Factors, by Demographic Characteristics 



N 

Total Percent of 
Students With One or 
Two Risk Factors 

Percent of Students With: 

One Risk Factor 
(GPA) 

One Risk Factor 
(Attendance) 

Two Risk 
Factors 

All Students 

3,382 

22.1% 

8.1% 

6.0% 

8.1% 

Female 

1,667 

19.5% 

5.6% 

6.6% 

7.3% 

Male 

1,715 

24.7% 

10.5% 

5.3% 

8.8% 







Ethnicity: 






Chinese 

1,302 

6.0% 

3.6% 

1.4% 

1.0% 

Latino 

696 

39.0% 

15.4% 

8.6% 

15.0% 

African-American 

428 

52.5% 

14.8% 

14.4% 

23.2% 

White 

268 

19.4% 

4.9% 

8.6% 

6.0% 

Filipino 

210 

18.6% 

7.1% 

6.2% 

5.2% 

Other Asian and 
Pacific Islander 

132 

14.4% 

3.8% 

4.9% 

5.7% 

All Other 

346 

18.7% 

6.5% 

5.6% 

6.5% 


Notes: The final three columns many not sum to the total percent due to rounding errors. A small number of students had missing 
data or declined to state parent's education. 

Exhibit 4 shows the number of students identified as at risk of not graduating who were enrolled in each of 
the fourteen major high schools. The number of identified students varied by the high school attended, 
from a low of 12 students at Lowell to a high of 100 students at Galileo. 5 This information can help school 
personnel, including principals, counselors, and other support staff, anticipate the number of incoming 
students who could benefit from additional support and monitoring, though the exact number of students 
identified at each high school will vary to year to year. 


5 Downtown, Ida B. Wells, and Independence high schools also serve SFUSD high school students, but few first-time 
ninth grade students. Newcomer did not serve high school students who were enrolled in SFUSD in eighth grade. Data 
from Gloria Davis, which served few students and is now closed, was removed. 
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ixhibit 4. Number of Ninth Grade Students With an Identified Risk Factor, by High School 
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High School Attended Fall Semester of 9th Grade (N=Total Number of Incoming SFUSD Eighth Grade Students) 


Exhibit 4 reflects the number of incoming ninth grade students who were also enrolled both semesters of eighth grade in SFUSD. 
The percent of the total incoming ninth grade class at each school that were also enrolled in SFUSD in eighth grade ranged from 
over 90% at Galileo and Washington to less than 70% at Mission and SOTA. 


Exhibit 5 uses the same data as Exhibit 4, but presents the percentage of the incoming class with zero, one, 
or two risk factors, instead of the raw number of students. There are four schools that are more clearly 
impacted by high-risk students: ISA, June Jordan, Mission, and O'Connell. Though these schools only 
serve between 74 and 160 ninth grade students (that were also enrolled in SFUSD in eighth grade the year 
before), approximately half of the incoming class was identified as having one or two risk factors. 

ixhibit 5. Percentage of Ninth Grade Students With an Identified Risk Factor, by High School 


■ Two Risk Factors ■ One Risk Factor (GPA) □ One Risk Factor (ATT) ■ Zero Risk Factors 



See notes for Exhibit 4. 
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Why Does Identifying Students Matter? 


Research is clear that the transition into high school is a critical time in students' academic careers, and that 
students who are successful in ninth grade are significantly more likely to graduate from high school 
(Allensworth & Easton, 2005). We also know that students enter high school with varying levels of 
preparation which affects their abilities to be successful academically. Identifying ninth grade students is 
an important strategy only if we believe there is an opportunity to positively affect a student's academic 
trajectory at this critical juncture. 

Exhibit 6 clearly illustrates that regardless of a student's risk factors, academic success in the first semester 
of ninth grade has a significant effect on long-term success in high school. Students who passed all their 
core courses in the first semester of ninth grade showed significantly higher graduation rates than students 
who failed even one core course, regardless of their pre-existing number of risk factors. Of the students 
with zero risk factors, 86.7% (349 out of 2,633) passed all their core courses first semester, but those who 
failed even one course had a four-year graduation rate that was 33 percentage points lower (56.0% 
compared to 88.8%). For those students with one risk factor, failing one or more core courses led to a 23 
percentage point drop in the likelihood of graduating; importantly, students with one risk factor were 
about evenly divided between those who passed all their core courses and those that failed one or more 
core courses. Passing all core courses for students with two risk factors more than doubled the percentage 
who graduated, but only 26.4% (72 out of 273) of these students were able to pass all their courses. 


Exhibit 6. Four-Year High School Graduation Rate, by Number of Risk Factors and Course Failures in 
the First Semester of 9 th Grade 



Notes: Core courses refer to Math, English, Social Studies, and Science. 


Of the students who entered with one risk, those with low attendance were more likely to pass all their 
core courses than students with low GPA; 62.4% of low attendance students (126 out of 202) passed all 
their core courses compared to 33.9% of low GPA students (93 out of 274). 

Selecting Students for Summer School Support 


The analysis above focuses on a set of indicators that can help principals and schools identify incoming 
students who may need additional supports early in their high school career. One strategy for helping 
these students is to enroll them in a transitional summer program between eighth and ninth grade, which 
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would require SFUSD to identify students much earlier than the end of eighth grade. Mirroring the 
approach taken above, we found two first semester eighth grade indicators that predicted a 25% four-year 
high school graduation rate: (1) GPA below 2.0 and (2) instructional attendance rate below 85%. This 
approach, which mirrors the end of the year eight grade indicators but with a lower attendance threshold, 
identified 633 students, slightly lower than the 749 identified by the year-end eighth grade indicators 
discussed above. 


Exhibit 7. Four-Year High School Graduation Rate, by Number of First-Semester Eighth Grade Risk 
Factors 



* Of the students with exactly one risk factor, 69.8% (n=316) had a low GPA and 30.2% (n=137) had a low attendance rate. 


Conclusion 


This report details the development of a parsimonious set of two indicators that can help SFUSD staff 
identify a substantial portion of the incoming ninth grade students who are at risk of not graduating within 
four years. Approximately two-thirds of these students are either African-American or Latino. This 
analysis makes clear that this initial identification should represent just a first step in monitoring student 
performance throughout high school. SFUSD and JGC are currently collaborating to develop additional 
indicators that can serve as best predictors of high school completion at the ninth grade level and beyond. 

Analysis in this report provides the foundation for an Early Warning System, but equally important as the 
data is the strategy for their dissemination and usage. Districts that have developed an EWS have also 
invested substantial time and resources to understand the best practices that can help these incoming 
students successfully transition into high school. In both raw numbers and percentage terms, the number 
of identified students is not equally divided among SFUSD high schools, and strategies to improve student 
performance may need to be substantially different among schools. 

High school graduation is an important first step for a student in reaching career goals, but research has 
made clear that too many students graduate high school unprepared for college-level work. Therefore, 
districts and schools also need to ensure that their students are taking the necessary steps to access college 
and develop the skills that will ensure that they are successful once they arrive. This may require 
additional indicators that are not currently collected, in part because they are not easily measured. 

SFUSD's implementation of ConnectEDU, a college and career planning tool that will be implemented at 
all SFUSD high schools next Fall through the Plan Ahead class, might serve as a complementary system to 
the EWS that provides continued monitoring of college access as students progress through high school 
and towards college. 
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